Contextual Sentence Decomposition

In this thesis, we introduce and study contextual sentence decomposition, which, intuitively, decomposes a given sentence into parts that semantically “belong together”. For example, a valid decomposition of the sentence “Usable parts of rhubarb include the edible stalks and the medicinally used roots, however its leaves are toxic” are the sub-sentences “Usable parts of rhubarb include the edible stalks”, “Usable parts of rhubarb include the edible stalks” and “however its leaves are toxic”. Our motivation for this problem comes from semantic full-text search. For a query plant edible leaves, semantic full-text search returns passages where instances of a plant, such as “rhubarb” (and not the word “plant”), are mentioned along with the words “edible” and “leaves”. One of the results this query might erroneously return is the original sentence above. With contextual sentence decomposition we avoid this false-positive, while at the same time maintaining the true factual contents of the original sentence. We propose two approaches for our problem, one based on a set of rules and one using machine learning. On a manually assembled ground truth, we achieve an F-measure of about 65 percent for the former and of 40 percent for the latter. For the semantic full-text search based on these approaches, evaluated on the English Wikipedia (27 GB of raw text), we achieve improvements nearly doubling the F-measure for some queries.